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Abstract 


This memo describes a detailed framework for composing and 
aggregating metrics (both in time and in space) originally defined by 
the IP Performance Metrics (IPPM), RFC 2330, and developed by the 
IETF. This new framework memo describes the generic composition and 
aggregation mechanisms. The memo provides a basis for additional 
documents that implement the framework to define detailed 
compositions and aggregations of metrics that are useful in practice. 


Status of This Memo 


This document is not an Internet Standards Track specification; it is 
published for informational purposes. 


This document is a product of the Internet Engineering Task Force 


(IETF). It represents the consensus of the IETF community. It has 
received public review and has been approved for publication by the 
Internet Engineering Steering Group (IESG). Not all documents 


approved by the IESG are a candidate for any level of Internet 
Standard; see Section 2 of RFC 5741. 


Information about the current status of this document, any errata, 


and how to provide feedback on it may be obtained at 
http: //www.rfc-editor.org/info/rfc5835. 
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1. Introduction 


The IP Performance Metrics (IPPM) framework [RFC2330] describes two 
forms of metric composition, spatial and temporal. The text also 
suggests that the concepts of the analytical framework (or A-frame) 
would help to develop useful relationships to derive the composed 
metrics from real metrics. The effectiveness of composed metrics is 
dependent on their usefulness in analysis and applicability to 
practical measurement circumstances. 


This memo expands on the notion of composition, and provides a 
detailed framework for several classes of metrics that were described 
in the original IPPM framework. The classes include temporal 
aggregation, spatial aggregation, and spatial composition. 


1.1. Motivation 


Network operators have deployed measurement systems to serve many 
purposes, including performance monitoring, maintenance support, 
network engineering, and reporting performance to customers. The 
collection of elementary measurements alone is not enough to 
understand a network’s behaviour. In general, measurements need to 
be post-processed to present the most relevant information for each 
purpose. The first step is often a process of "composition" of 
single measurements or measurement sets into other forms. 
Composition and aggregation present several more post-processing 
opportunities to the network operator, and we describe the key 
motivations below. 


1.1.1. Reducing Measurement Overhead 


A network’s measurement possibilities scale upward with the square of 
the number of nodes. But each measurement implies overhead, in terms 
of the storage for the results, the traffic on the network (assuming 
active methods), and the operation and administration of the 
measurement system itself. In a large network, it is impossible to 
perform measurements from each node to all others. 


An individual network operator should be able to organize their 
measurement paths along the lines of physical topology, or routing 
areas/Autonomous Systems, and thus minimize dependencies and overlap 
between different measurement paths. This way, the sheer number of 
measurements can be reduced, as long as the operator has a set of 
methods to estimate performance between any particular pair of nodes 
when needed. 
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Composition and aggregation play a key role when the path of interest 
spans multiple networks, and where each operator conducts their own 
measurements. Here, the complete path performance may be estimated 
from measurements on the component parts. 


Operators that take advantage of the composition and aggregation 
methods recognize that the estimates may exhibit some additional 
error beyond that inherent in the measurements themselves, and so 
they are making a trade-off to achieve reasonable measurement system 
overhead. 


1.1.2. Measurement Re-Use 


There are many different measurement users, each bringing specific 
requirements for the reporting timescale. Network managers and 
maintenance forces prefer to see results presented very rapidly, to 
detect problems quickly or see if their action has corrected a 
problem. On the other hand, network capacity planners and even 
network users sometimes prefer a long-term view of performance, for 
example to check trends. How can one set of measurements serve both 
needs? 


The answer lies in temporal aggregation, where the short-term 
measurements needed by the operations community are combined to 
estimate a longer-term result for others. Also, problems with the 
measurement system itself may be isolated to one or more of the 
short-term measurements, rather than possibly invalidating an entire 
long-term measurement if the problem was undetected. 


1.1.3. Data Reduction and Consolidation 


Another motivation is data reduction. Assume there is a network in 
which delay measurements are performed among a subset of its nodes. 

A network manager might ask whether there is a problem with the 
network delay in general. It would be desirable to obtain a single 
value that gives an indication of the overall network delay. Spatial 
aggregation methods would address this need, and can produce the 
desired "single figure of merit" asked for, which may also be useful 
in trend analysis. 


The overall value would be calculated from the elementary delay 
measurements, but it is not obvious how: for example, it may not be 
reasonable to average all delay measurements, as some paths (e.g., 
those having a higher bandwidth or more important customers) might be 
considered more critical than others. 
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Metric composition can help to provide, from raw measurement data, 
some tangible, well-understood and agreed-upon information about the 
service guarantees provided by a network. Such information can be 
used in the Service Level Agreement/Service Level Specification 
(SLA/SLS) contracts between a service provider and its customers. 


1.1.4. Implications on Measurement Design and Reporting 


If a network measurement system operator anticipates needing to 
produce overall metrics by composition, then it is prudent to keep 
that requirement in mind when considering the measurement design and 
sampling plan. Also, certain summary statistics are more conducive 
to composition than others, and this figures prominently in the 
design of measurements and when reporting the results. 


2. Requirements Language 


The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 
document are to be interpreted as described in RFC 2119 [RFC2119]. 


3. Purpose and Scope 


The purpose of this memo is to provide a common framework for the 
various classes of metrics that are composed from primary metrics. 
The scope is limited to the definitions of metrics that are composed 
from primary metrics using a deterministic function. Key information 
about each composed metric is included, such as the assumptions under 
which the relationship holds and possible sources of 
error/circumstances where the composition may fail. 


At this time, the scope of effort is limited to composed metrics for 
packet loss, delay, and delay variation, as defined in [RFC2679], 
[RFC2680], [RFC2681], [RFC3393], [RFC5481], and the comparable 
metrics in [Y.1540]. Composition of packet reordering metrics 
[RFC4737] and duplication metrics [RFC5560] are considered research 
topics at the time this memo was prepared, and are beyond the scope 
of this document. 


This memo will retain the terminology of the IPPM Framework [RFC2330] 
as much as possible, but will extend the terminology when necessary. 
It is assumed that the reader is familiar with the concepts 
introduced in [RFC2330], as they will not be repeated here. 
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4. Terminology 


This section defines the terminology applicable to the processes of 
metric composition and aggregation. 


4.1. Measurement Point 


A measurement point is the logical or physical location where packet 
observations are made. The term "measurement point" is synonymous 
with the term "observation position" used in [RFC2330] when 
describing the notion of wire time. A measurement point may be at 
the boundary between a host and an adjacent link (physical), or it 
may be within a host (logical) that performs measurements where the 
difference between host time and wire time is understood. 


4.2. Complete Path 
The complete path is the actual path that a packet would follow as it 
travels from the packet’s Source to its Destination. A complete path 
may span the administrative boundaries of one or more networks. 

4.3. Complete Path Metric 
The complete path metric is the Source-to-Destination metric that a 
composed metric attempts to estimate. A complete path metric 


represents the ground-truth for a composed metric. 


4.4. Complete Time Interval 


The complete time interval is comprised of two or more contiguous 
sub-intervals, and is the interval whose performance will be 
estimated through temporal aggregation. 


4.5. Composed Metric 


A composed metric is an estimate of an actual metric describing the 
performance of a path over some time interval. A composed metric is 
derived from other metrics by applying a deterministic process or 
function (e.g., a composition function). The process may use metrics 
that are identical to the metric being composed, or metrics that are 
dissimilar, or some combination of both types. 


4.6. Composition Function 
A composition function is a deterministic process applied to 


individual metrics to derive another metric (such as a composed 
metric). 
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4.7. Ground Truth 


As applied here, the notion of "ground truth" is defined as the 
actual performance of a network path over some time interval. The 
ground truth is a metric on the (unavailable) packet transfer 
information for the desired path and time interval that a composed 
metric seeks to estimate. 


4.8. Interval 
An interval refers to a span of time. 
4.9. Sub-Interval 


A sub-interval is a time interval that is included in another 
interval. 


4.10. Sub-Path 


A sub-path is a portion of the complete path where at least the 
sub-path Source and Destination hosts are constituents of the 
complete path. We say that such a sub-path is "involved" in the 
complete path. 


Since sub-paths terminate on hosts, it is important to describe how 
sub-paths are considered to be joined. In practice, the Source and 
Destination hosts may perform the function of measurement points. 


If the Destination and Source hosts of two adjoining paths are 
co-located and the link between them would contribute negligible 
performance, then these hosts can be considered equivalent (even if 
there is no physical link between them, this is a practical 
concession). 


If the Destination and Source hosts of two adjoining paths have a 
link between them that contributes to the complete path performance, 
then the link and hosts constitute another sub-path that is involved 
in the complete path, and should be characterized and included in the 
composed metric. 


4.11. Sub-Path Metrics 
A sub-path path metric is an element of the process to derive a 


composed metric, quantifying some aspect of the performance of a 
particular sub-path from its Source to Destination. 
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5. Description of Metric Types 


This section defines the various classes of composition. There are 
two classes more accurately described as aggregation over time and 
space, and the third involves concatenation in space. 


5.1. Temporal Aggregation Description 


Aggregation in time is defined as the composition of metrics with the 
same type and scope obtained in different time instants or time 
windows. For example, starting from a time series of the 
measurements of maximum and minimum one-way delay (OWD) on a certain 
network path obtained over 5-minute intervals, we obtain a time 
series measurement with a coarser resolution (60 minutes) by taking 
the maximum of 12 consecutive 5-minute maxima and the minimum of 12 
consecutive 5-minute minima. 


The main reason for doing time aggregation is to reduce the amount of 
data that has to be stored, and make the visualization/spotting of 
regular cycles and/or growing or decreasing trends easier. Another 
useful application is to detect anomalies or abnormal changes in the 
network characteristics. 


In RFC 2330, the term "temporal composition" is introduced and 
differs from temporal aggregation in that it refers to methodologies 
to predict future metrics on the basis of past observations (of the 
same metrics), exploiting the time correlation that certain metrics 
can exhibit. We do not consider this type of composition here. 


5.2. Spatial Aggregation Description 


Aggregation in space is defined as the combination of metrics of the 
same type and different scope, in order to estimate the overall 
performance of a larger network. This combination may involve 
weighing the contributions of the input metrics. 


Suppose we want to compose the average one-way delay (OWD) 
experienced by flows traversing all the origin-destination (OD) pairs 
of a network (where the inputs are already metric "Statistics"). 
Since we wish to include the effect of the traffic matrix on the 
result, it makes sense to weight each metric according to the traffic 
carried on the corresponding OD pair: 


OWD_sum=f1*OWD_1+f£2*OWD_2+...+f£n*OWD_n 


where fi=load_OD_i/total_load. 
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A simple average OWD across all network OD pairs would not use the 
traffic weighting. 


Another example metric that is "aggregated in space" is the maximum 
edge-to-edge delay across a single network. Assume that a Service 
Provider wants to advertise the maximum delay that transit traffic 
will experience while passing through his/her network. There can be 
multiple edge-to-edge paths across a network, and the Service 
Provider chooses either to publish a list of delays (each 
corresponding to a specific edge-to-edge path), or publish a single 
maximum value. The latter approach simplifies the publication of 
measurement information, and may be sufficient for some purposes. 
Similar operations can be provided to other metrics, e.g., "maximum 
edge-to-edge packet loss", etc. 


We suggest that space aggregation is generally useful to obtain a 
summary view of the behaviour of large network portions, or of 
coarser aggregates in general. The metric collection time instant, 
i.e., the metric collection time window of measured metrics, is not 
considered in space aggregation. We assume that either it is 
consistent for all the composed metrics, e.g., compose a set of 
average delays all referring to the same time window, or the time 
window of each composed metric does not affect the aggregated metric. 


5.3. Spatial Composition Description 


Concatenation in space is defined as the composition of metrics of 
same type with (ideally) different spatial scope, so that the 
resulting metric is representative of what the metric would be if 
obtained with a direct measurement over the sequence of the several 
spatial scopes. An example is the sum of mean OWDs of adjacent edge- 
to-edge networks, where the intermediate edge points are close to 
each other or happen to be the same. In this way, we can for example 
estimate OWD_AC starting from the knowledge of OWD_AB and OWD_BC. 
Note that there may be small gaps in measurement coverage; likewise, 
there may be small overlaps (e.g., the link where test equipment 
connects to the network). 


One key difference from examples of aggregation in space is that all 
sub-paths contribute equally to the composed metric, independent of 
the traffic load present. 


5.4. Help Metrics 
In practice, there is often the need to compute a new metric using 
one or more metrics with the same spatial and time scope. For 


example, the metric rtt_sample_variance may be computed from two 
different metrics: the help metrics rtt_square_sum and the rtt_sum. 
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The process of using help metrics is a simple calculation and not an 
aggregation or a concatenation, and will not be investigated further 
in this memo. 


5.5. Higher-Order Composition 


Composed metrics might themselves be subject to further steps of 
composition or aggregation. An example would be the delay of a 
maximal path obtained through the spatial composition of several 
composed delays for each complete path in the maximal path (obtained 
through spatial composition). All requirements for first-order 
composition metrics apply to higher-order composition. 


An example using temporal aggregation: twelve 5-minute metrics are 
aggregated to estimate the performance over an hour. The second step 
of aggregation would take 24 hourly metrics and estimate the 
performance over a day. 


6. Requirements for Composed Metrics 


The definitions for all composed metrics MUST include sections to 
treat the following topics. 


The description of each metric will clearly state: 

1. the definition (and statistic, where appropriate); 

2. the composition or aggregation relationship; 

3. the specific conjecture on which the relationship is based and 
assumptions of the statistical model of the process being 


measured, if any (see [RFC2330], Section 12); 


4. a justification of practical utility or usefulness for analysis 
using the A-frame concepts; 


5. one or more examples of how the conjecture could be incorrect and 
lead to inaccuracy; 


6. the information to be reported. 


For each metric, the applicable circumstances will be defined, in 
terms of whether the composition or aggregation: 


o Requires homogeneity of measurement methodologies, or can allow a 
degree of flexibility (e.g., active or passive methods produce the 
"same" metric). Also, the applicable sending streams will be 
specified, such as Poisson, Periodic, or both. 
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o Needs information or access that will only be available within an 
operator’s network, or is applicable to inter-network composition. 


o Requires precisely synchronized measurement time intervals in all 
component metrics, or perhaps only loosely synchronized time 


intervals, or has no timing requirements at all. 


o Requires assumption of component metric independence with regard 
to the metric being defined/composed, or other assumptions. 


o Has known sources of inaccuracy/error and identifies the sources. 
6.1. Note on Intellectual Property Rights (IPR) 
If one or more components of the composition process are encumbered 
by Intellectual Property Rights (IPR), then the resulting composed 
metrics may be encumbered as well. See BCP 79 [RFC3979] for IETF 
policies on IPR disclosure. 
7. Guidelines for Defining Composed Metrics 


7.1. Ground Truth: Comparison with Other IPPM Metrics 


Figure 1 illustrates the process to derive a metric using spatial 
composition, and compares the composed metric to other IPPM metrics. 


Metrics <M1, M2, M3> describe the performance of sub-paths between 
the Source and Destination of interest during time interval <T, Tf>. 
These metrics are the inputs for a composition function that produces 
a composed metric. 
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Sub-Path Metrics 
++ M1 ++ ++ M2 ++ ++ M3 ++ 


cree | ee Midas A eee || Dst 
++ `. ++ t+ | ++ Ht.’ ++ 
NN | r 
Nic: | , 
~ r 
—-' e pis 2 ve 
f- 
r ~ 
r 
| Composition 
\ Function 2 
~ = apt 
Me... --! 
| 
++ | ++ 
Src | | sect ele iede et ed as larie | | Dst 
++ Composed Metric ++ 
++ Complete Path Metric ++ 
Src | | AAA ver eee Ba ede | | Dst 
++ ++ 
Spatial Metric 
++ s1 ++ S2 ++ s3 ++ 
en TO Ihsa eten Mesas || Dst 
++ ++ ++ ++ 


Figure 1: Comparison with Other IPPM Metrics 


The composed metric is an estimate of an actual metric collected over 
the complete Source-to-Destination path. We say that the complete 
path metric represents the ground truth for the composed metric. In 
other words, composed metrics seek to minimize error with regard to 
the complete path metric. 


Further, we observe that a spatial metric [RFC5644] collected for 
packets traveling over the same set of sub-paths provides a basis for 
the ground truth of the individual sub-path metrics. We note that 
mathematical operations may be necessary to isolate the performance 
of each sub-path. 


Next, we consider multiparty metrics (as defined in [RFC5644]) and 
their spatial composition. Measurements to each of the receivers 
produce an element of the one-to-group metric. These elements can be 
composed from sub-path metrics, and the composed metrics can be 
combined to create a composed one-to-group metric. Figure 2 
illustrates this process. 
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Sub-Path Metrics 


++ MI ++ ++ M2 ++ ++ M3 ++ 
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++ 


One-to-Group Metric 


++ ++ ++ ++ 
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++ ++. ++ ++ 

Ve 
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ve 

`- ++ 
`=. | |Revr3 

++ 


Figure 2: Composition of One-to-Group Metrics 


Here, sub-path metrics M1, M2, and M3 are combined using a 
relationship to compose the metric applicable to the Src-Revrl path. 
Similarly, M1, M4, and M5 are used to compose the Src-Rcvr2 metric 
and M1, M4, and M6 compose the Src-Revr3 metric. 


The composed one-to-group metric would list the Sre-Revr metrics for 
each receiver in the group: 


(Composed-Revrl, Composed-Revr2, Composed-Rcvr3) 
The ground truth for this composed metric is of course an actual one- 
to-group metric, where a single Source packet has been measured after 
traversing the complete paths to the various receivers. 

7.1.1. Ground Truth for Temporal Aggregation 
Temporal aggregation involves measurements made over sub-intervals of 
the complete time interval between the same Source and Destination. 


Therefore, the ground truth is the metric measured over the desired 
interval. 
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7.1.2. Ground Truth for Spatial Aggregation 


Spatial aggregation combines many measurements using a weighting 
function to provide the same emphasis as though the measurements were 
based on actual traffic, with inherent weights. Therefore, the 
ground truth is the metric measured on the actual traffic instead of 
the active streams that sample the performance. 


7.2. Deviation from the Ground Truth 


A metric composition can deviate from the ground truth for several 
reasons. Two main aspects are: 


o The propagation of the inaccuracies of the underlying measurements 
when composing the metric. As part of the composition function, 
errors of measurements might propagate. Where possible, this 
analysis should be made and included with the description of each 
metric. 


o A difference in scope. When concatenating many active measurement 
results (from two or more sub-paths) to obtain the complete path 
metric, the actual measured path will not be identical to the 
complete path. It is in general difficult to quantify this 
deviation with exactness, but a metric definition might identify 
guidelines for keeping the deviation as small as possible. 


The description of the metric composition MUST include a section 
identifying the deviation from the ground truth. 


7.3. Incomplete Information 


In practice, when measurements cannot be initiated on a sub-path or 
during a particular measurement interval (and perhaps the measurement 
system gives up during the test interval), then there will not be a 
value for the sub-path reported, and the result SHOULD be recorded as 
"undefined". 


7.4. Time-Varying Metrics 


The measured values of many metrics depend on time-variant factors, 
such as the level of network traffic on the Source-to-Destination 
path. Traffic levels often exhibit diurnal (or daily) variation, but 
a 24-hour measurement interval would obscure it. Temporal 
aggregation of hourly results to estimate the daily metric would have 
the same effect, and so the same cautions are warranted. 
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Some metrics are predominantly* time-invariant, such as the actual 
minimum one-way delay of a fixed path, and therefore temporal 
aggregation does not obscure the results as long as the path is 
stable. However, paths do vary, and sometimes on less predictable 
time intervals than traffic variations. (* Note: It is recognized 
that propagation delay on transmission facilities may have diurnal, 
seasonal, and even longer-term variations.) 


8. Security Considerations 


The security considerations that apply to any active measurement of 
live networks are relevant here as well. See [RFC4656] and 
[RFC5357]. 


The exchange of sub-path measurements among network providers may be 
a source of concern, and the information should be sufficiently 
anonymized to avoid revealing unnecessary operational details (e.g., 
the network addresses of measurement devices). However, the schema 
for measurement exchange is beyond the scope of this memo and likely 
to be covered by bilateral agreements for some time to come. 
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